EuropSisches 
Patentamt 



European 
Patent Office 



Office europeen 
des brevets 




Bescheinigung Ceiiificate 



Attestation 



Die angehefteten Unterla- 
gen stimmen mit der 
ursprOnglich eingereichten 
Fassung der auf dem nSch- 
sten Blatt bezeichneten 
europdischen Patentanmel- 
dung Qberein. 



The attached documents 
are exact copies of the 
European patent application 
described on the following 
page, as originally filed. 



Les docunfkents fix^s d 
cette attestation sont 
confbrmes ^ la version 
initialement d^poste de 
la demande de brevet 
europ6en sp6cifite k la 
page suivante. 



Patentanmeidung Nr. Patent application No. Demande de brevet n** 

03103971.2 



m 

> 

i 

CP 

m 
o 

o 



CERTIFIED COPY Of 
PRIORITY DOCUMENT 



PRIORITY DOCUMENT 

SUBMITEED OR TRANSMITTED IN 
COMPLIANCE WITH 
RULE 17.1(a) OR (b) 



Der Prdsident des EuropSlschen Patentamts; 
tm Auftrag 

For the President of the European Patent OfRce 
Le President de roffice europ^n des brevets 

P.O. 



R C van Dlllc 




Europalsches 
Patenfamt 



European 
Patent Office 



Office europeen 
des brevets 



Anmeldung Nr: 

Application no.: 03103971.2 
Demande no: 



Anmeldetag: 

Date of filing: 27-10.03 
Date de dSpot: 



Anmel der/AppI 1cant( s)/D8inandeurC s) : 

Konlnklijke Philips Electronics N.V- 
Groenewoudseweg 1 
5621 BA Eindhoven 
PAYS-BAS 



Bezelchnung der Erf Indung/Tltle of the Inventlon/Tltre de T Invention: 
(Falls die Bezelchnung der Erflndung nicht angegeben ist, slehe Beschrel bung. 
If no title Is shown please refer to the description. 
SI aucun titre n'est 1nd1qu4 se referer a la description.) 

Screen-wise presenting of search results 



In Anspruch genominene Pr1or1St(en} / Prlorlty(les) claimed /Pr1or1t6(s) 
revend1qu6e(s) 

Staat/Tag/Aktenzelchen/State/Date/Flle no. /Pays/Da te/Num^ro de d^pdt: 



Internationale Patentklassif Ikatlon/Internatlonal Patent Classification/ 
Classification Internationale des brevets: 

G06F17/30 



Am Anmeldetag benannte Vertragstaa ten/Contracting states designated at date of 
flllng/Etats contractants dSslgn^es lors du d^pdt: 

AT BE BG CH CY CZ DE DK EE ES FX FR 6B 6R HU IE IT LU MC NL 
PT RO SE SI SK TR LI 



03103971.2 

EPA/EPO/OEB Form 1014.2 - 01.2000 7001014 



2 



PHNL031271EPP 

1 27.10.2003 

Scieen-Mdse presenting of search results 



The invention relates to a mediod of pres^ting results of searching a 
document, the search result comprises a hit in the document The invention also relates to a 
^aratus for presenting results of searching a document and a conq)uter program product 



Document EP0596247 discloses a method of searching heywords in text 
information. A user can initiate a search using a search string to find occurrences of the 
search string fiom a particular document A computer produces a hit list of documents and 
pages that contain the search string. Once the user selects a page on the hit lisl^ the computer 
displays tiie page and hig^ghts hits within fbe selected page. Hie user may use navigation 
kq^ to move to the nejd; lut or tiie pluvious hit displayed on the 

Occurrences of Ae search string can be distributed arbitrarity in the document 
Some of the occurrences may be clustered together, e.g. in a section of tiie document where 
the search string occur often. In another example, a particular section of the document may 
conqirise only few occurrences. 

If the user uses known '*fid next" or "find previous" commands to browse 
flMough tiie occurrences, it may be difficult for tiie user to determine wheflier a particular 
occurrence is in tiie current cluster or in a different part of flie document If tiie user simply 
scrolls the document to see the occurrences, it may be very time consuming. Thus, tiie known 
method is ine£5cient to present occurrences to the user. 



It is an object of the present invention to obviate the drawbacks of the prior art 
search metiiods, and to provide a mefliod of presenting results of searching a document, 
which enables the user to navigate easily and effidenfly between the search results. 

This object is realized in that tiie metiiod conqsrises the steps of: 

presenting at least one hit in a part of tiie document displayed on a scteen, and 
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subsequenfly presenting at least one further hit in a further part of the 
document displayed on the same screen, at least one further hit being not comprised in said 
part of tiie document v^^ch has been displayed. 

Generally, many documents are too large to be viewed entirely on a single 
screen without sacrificing readability. A common element of most browsing systems is that a 
document can be viewed screen-wise, i.e. a part of flie document is shown on the screen at a 
time. The part of the documrait displayed on flie sosen may be a paragraph, several 
paragr^hs, a page or several pages of said document. For example, the page may comprise a 
number of lines and columns of a text document 

All search hits, i.e. occurrences found upon the search, which are currently on 
the screen are presented to the user, e.g. by visually hi^ghting using some color, so that 
Ihey are shnuHaneously presented to the user. Of course, it may h^en that only one hit is 
fbund and presented on the screen. 

To display further occurrence or occurrences in the document, which are not 
shown on the current sa»en, the document may be browsed screen-wise, Le. step-wise 
wherdn one screen is shown at each step. Thus, a next screen wifli the finrtfaer part of the 
document is displayed. The further part of Ae document comprises at least one finiher hit in 
said furflier iwrt of the document ^^ch has not yet been presented. 

For example, in contrast to lbs known text browsing methods, a "find nextf' 
command will not start searching for and higUighting the first next occurrence, but, mstead, 
the furttier page comprising the occurrence or the occurrences vrtddi have not yet been 

presented is displayed. 

According to the invention, as few screens as possible are displayed in order to 
present the search results, i.e. screen-wise navigated, and the user is provided wilh an 
overview of all search results txpon each screen. The user gets a clear overview of the 
distribution of occurrences, and recognizes chisters at a glance. For example, the repeated 
find-commands do not result in unpredictable small junaps on the current screen, but show 
only occurrences of the search string which the user has not seen yet A few **find next" 
commands may be sufficient to find all occurrences of the search string in the document This 
is especially usefiil fbr a device with a limited user interfiice, e.g. a mobile phone, portable 
computer, a remote control unit, etc. The user can browse Ihrou^ the search results with few 
commands. 

The object of the pr^ent invention is also realized in that the invention 
provides for an appaxatas for presenting results of searching a document, tiie search results 
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comprise at least one hit in the document; Ihe qyparatus conqnising a display means coupled 
to a processor for enabling ibe apparatus: 

to present at least one hit in a part ofthe document displayed on a screen, and 
to present subsequently at least one further hit in a &rther part of the 
5 document displayed on the same screen, at least one fiirther hit being not coroprised in said 

pait of flie document which has been disfplayed. 

The apparatus is arranged to function as described above with reference to the 

method of the present invention. 

10 

These and other aspects of the invention will be &rther explained and 
described with reference to the following drawings: 

Fig. 1 shows an embodiment of the method of the present invention; 

Figs. 2 and 3 show a diagram illustrating examples of screens displaying parts 
IS of a document with found search results; 

Fig. 4 shows a functional block diagram of an embodiment of an ^yparatus 
according to the invention. 

20 Fig. 1 shows an embodiment of the method of presenting search results of a 

document according to the present invention. Generally, the invention can be used to present 
results of searching any information which can be searched using, for example, a computer, 
and displayed or pr^ented otiierwise to a user. In this context, a "document " may comprise 
any combination of information such as text, video images, photos, graphics, audio data and 

25 other digital data such, e.g., meta-data according to the MPEG-7 standard which m^ be used 
to describe and search digitized materials by means of sampling, as well as by using lexical 
search terms. The audio datamay be in formats like theMPEG-2 standard (Moving Picture 
Experts Group), AVI (Audio \^deo Interleave) formal WMA (Windows Media Audio) 
format, etc. The video datamay be in formats like GIF (Graphic Interchange Format), JPEG 

30 (named after the Joint Photographic Experts Groiq>), MPEG-4, ete. The text mformation may 
be in the ASCII (American Standard Code for Information Interchange) formal PDF (Adobe 
Acrobat Format) format, HTML (HyperTextMaikiq) Language) format, for exanqile. As 
another example, the meta-data may be in the XML (Extensible Markup Language) format. 
Thus, the wording ^'documenf ' is not limited to text documents. 
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A search query for searching the document may first be obtained at step 1 10. 
In text documents, flie search may be initiated with a dialogue window for inputting the 
search string and various search options. 

Using the query, the search is p^ormed at step 120. There are many known 
methods of searching documents on the basis of the text query. Upon the search, occunences 
of the search string may be found. One of the methods is described in EP0596247, where 
images are analyzed to obtain a text index which is then searched as conventional text 
informatioiL The same or similar techniques as for searching flie text information may be 
appUed for searching the meta-data or data stmctures. 

At step 130, a part of the document comprising at least one found occurrence, 
i.e. at least one hit, is displayed on a screen. The part of the document is entirely displayable 
on the screen so that the user can see content of said part. 

The hits in the displayed part of the document may subsequentiy be presented 
to tiie user in different ways at step 1 40. For exanq[>le, the search strings found in the text may 
be visually highlighted. 

At step 1 50, it may be chected whettier there are any forther hits, i.e. the 
further search rraults, in ibe document which have not been presented yet If tiie forther hit or 
hits successive witii respect to the part of the document which has been displayed are found 
(e.g., after a subsequent search), a further part of the documents with the forther search 
results may be selected at step 1 60. For example, the beg^rming and the end of the further part 
are determmed. The wording "successive" means that the further occurrences may not be 
comprised in the part of the document which has been displayed. 

A command for displaying the further part of the document may be awaited 
ftom flie user at step 170. Alternatively, the command may be generated automatically and 
the user input may be dispensed with. 

At step 1 80, the forther part of the document with the further search results 
may be displayed, and, at step 190, foe further search results may be presented, e.g., visually 
hi^ghted, as described above with reference to steps 130 and 140. 

Steps ISO to 190 may iterate if otho: further occunences are found in foe 

document 

An embodiment of foe mefood of foe present invention is explained with 
reference to Fig. 2. Fig. 2 shows a screen 2 1 0 and a screen 220 in vMch a forst part of foe 
document 200 and a second part of foe document 200, i.e. foe forther part of foe document, 
are respectively shown. The first screen 210 comprises occurrences of a fost text string 230. 
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Hie second screen 220 con^irises occurrences of Ihe same fbrst text string 230 and 
occunences of a second text string 240. 

It shoidd be noted that the part of die document which is displayed may be 
diEferent fix)m a logical page, e.g. , a conventional '*page" in a text MS Word document For 
5 example, this logical page may merely indicate a part of the document which is intended for 
printing. 

First, the screen 210 is displayed with the occurrences of die first siring 230. 
Then, die second screen 220 is subsequently displayed with the further occurrences of the 
first string 230 and the second string 240. The second screen 220 excludes the occurrences in 
10 the first screen 210. The screens 210 and 220 may be displayed on the same device, in the 
same area. 

Other parts of the document 200 which do not comrprise any search results are 
not displayed. The user may be provided with all search results in two screens displayed 
automatically or upon the user command, e.g., the **find nexf * command. Normally, each 

15 occurrence is prraented only.once, except in some special circumstances, for example, when 
the end of the document is reached. 

The screen 210 may be aligned, widi respect to the found occunences in the 
corresponding part of die document, so that a maximimi of the occurrences is shown on the 
same screen. For instance. Fig. 2 shows that the screen 2 10 is aligned to present one 

20 occurrence 230 at the top of the screen. As a result, the other occurrence is presented at the 
bottom of the screen 210. This would not be possible if the occurrences at the top was, for 
example, in the middle of the screen 210. Generally, the aligmnent of the screen may be 
performed in any direction depending on the document, e.g., upward, downward, left or right 
In one of the embodiments of the present invention, there are two or more 

25 search queries, for example, the search text strings 230 and 240 as shown in Fig. 2. The 
srarch results for one or more of the search queries may be presented simultaneously. For 
instance, the search results for only one search query may be presented, whereas the search 
results for the other queries may be shown as long as they occur on the same screen with Ihe 
fomier results . A part of the document to be displayed may be selected on the basis of this 

30 choice of whether to present search results for one or more of die search queries. For 
example, in Fig. 2, the screen 220 would be shifted (aligned) to a position 250 if only 
occurrences for the search string 240 are to be shown. 

Certain documents may have a complex structure such as a table, a tree- 
structure, etc. Tree structured documents may be used in EFG (Electronic Programme Guide) 
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systems, TV lecommenders (e-g- genie hierarchies), file directories in audio jukeboxes and 
cameras, database reports, etc. In this case, the search results can not be viewed just by 
scrolling the document fix>m top to bottom, for example. Multiple scrolling direction may be 
required, for example, m tihie table or the tree-like stracture where three branches extend 
5 beyond the screen boundaries in horizontal or vertical directions. This may cause that the 
document is scrolled alternately in different directions, e.g., upward, downward, left or rig^t 
This may be confusing for the person viewing Hxe document 

An exaxi^>le of applying the method according to the present invention to the 
document comprising 1able-lils» information 300 is explained with reference to Fig. 3. Fig. 3 
10 shows screens 310, 320 and 330 in which corresponding parte of flie document 300 are 

displayed Other parts of the document may be skipped and not shown to the user. The hite 
are shown only once in the respective screens. The screens are not adjacent, as it would be 
done in conventional systems, but positioned to show the search results according to the 
method of the invention. 
15 It may h^en that in tables like table 300, the text in cells cannot be shown 

completely and is hidden or truncated. However, occurrences may be feund in such cells, and 
such occurrences may not be visible because they are in the hidden part of the text in the celL 
This has the disadvantage tiiat it is not clear at first sight which part of the cell matches the 
search string. To solve this problem, the occurrences may be shown in the following manner. 
20 A first part, which is visible, of the text in ike cell may be shown if the search string occurs 
within the first part. If the occurrences are in the hidden part of the text of the cell, a part of 
the text in which the occurrences are found is shown, and the otfa^ part of the text may be 
skq)ped (not shown). The sfcqiped part of the text in the cell may be represented by sfpedal 
symbols like etc. 

25 A fiirfher enibodiment of the present invention relates to documents which 

may be edited. Known text processors allow for replacing one &imd occurrence at a time. 
When a particular occurrence is highlighted on the screen, a subsequent typing, press of a 
button or the like may cause tiie highlighted occurrence to be replaced by input characters. It 
also known to replace all occurrences in the whole document Thus, flie user may be required 

30 to act on every occurrence found in the document, which is time-consuming or to replace the 
occurrences in the whole document which is not desirable because flie user cannot see flie 
whole document To solve this problem, the method of the preset mvention may provide for 
replacing all occurrences on the current screen by a single command, press of the button, etc. 
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Fig. 4 shows a block diagram schematically illustrating an apparatus 400 for 
presenting results of searching a document The ^)paratus may comprise a central processing 
unit (CPU) 410, a memory 420, an optional ii^ut unit 430 and a display device 440. 

The display device may be a monitor such as a conventional CRT, or any otiier 
S device arranged to disfplayingtiie document and the search results on the screen. The user 
inpxA unit may be a keyboard, a pointer control device such a computer mouse, etc. The input 
unit may be equipped with cursor control keys, for example, a LEFT key, a RIGHT key, an 
UP key and a DOWN key. In another example^ the input unit may be combined with the 
disfplay device and comprise a touch sensitive screen. In a further example, the ioput unit may 
10 comprise a microphone (not shown) and a speech recognition facility, typically unplemented 
as a software program to be executed by the CPU. The display device may be coupled to 
. sfpeakers (not shown) for reprodudng the audio information. 

The memory, e.g., a conventional Random Access Memory (ElAM), may be 
arranged to store a computer program to be executed by the CPU for enabling the CPU to 
IS function as described above witii reference to the method of the present invention. The 

memory may also be arranged to store the document and the CPU may be arranged to access 
the document stored in the memory for perfcmning the search. The CPU may be a g^eral 
purpose microprocessor unit It will be clear to the skilled person how to implement the 
present invention in the cq>paratus. 
20 The CPU may be coupled to a conimunication unit (not shown) arranged to 

obtain the document fix>m an external source. For example, the communication unit may be a 
well-known modem arranged for connecting to the bitemet, a communication port &r 
obtaining the document from a scaimer. 

It should be understood that the present invention is not r^tricted to a 
25 particular embodiment shown in Figure 4 but may also be implemented in a TV set; a home 
dnema system, a portable video player, a remote control imit or a mobile phone, as well as in 
<a conventional conqniter. 

The various program products may iniplement the functions of the device and 
method of the present invention and may be combined in several ways with tiie hardware or 
30 located in different other devices. Variations and modifications of the described embodimmt 
are possible within the scope of the inventive concept 

For instance, the method of the present invention is exphdned above witii the 
exaixq>les referring to the text search in documents. However, the invention may be ^>plied 
regarding audio and video information. 
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The search query may also be an audio query, video query or any combination 
thereof with the text query. The video information may be searched using different methods. 
For example, video data to be searched may be pre-maiked using tags, so-called meta-data 
using, for exarnple, the MPEG-7 standard in the known manner. Such video data may be 

5 searched in a conventional way, e.g., using keywords. 

In another example, the video iixformation may be searched by s^lying 
various video analyses. Some methods of video analyses include steps such as segmentation 
of the video data, classification, and recognition, for example, recognition of frontal views of 
human faces. Other methods obtain the video query, e.g., a video clip or a still image, of a 

10 size much smaller than the searched video information to find parts of the video information 
that match the video query, using special algorithms wilh some measure of quality of match. 
For example, the algorithms may utilize imagQ similarity measures after the video images are 
split into blocks, or video similarity measures to measure similarity of clips. These known 
methods are applicable to searching video databases. 

1 s Many methods of searching audio information are known. The metadata may 

be used to tag audio information (such as title, date of recording, subject, or person). Upon a 
specific text search query, specific data fiiom the searched audio information may be 
retrieved. In another metiiod, the audio information may be converted to text, that can be 
searched very quickly for occurrences of a specified keyword or keywords. In another 

20 retrieval technique, the keyword may be presented by audio parameters for performing ibc 
search directly on the audio information. 

Other techniques provide for using pre-processing algorithms to describe the 
predetermined characteristics of the audio information, for example, representing a phonetic 
content of the audio information. During the preprocessing, auxiliary data is created that is 

25 subsequentiy searched when a search query is input. If the s^rch query is the text, it may be 
converted or represented by search audio parameters that are used to search the auxiliary 
data. The seardi qu^ may also be audio data which are analyzed to obtain similar search 
audio parameters. 

The hits found in the part of the video information or audio information may 
30 subsequentiy be presented to the user in different ways. For example, the image found in the 
video information may be provided with a border or edg^ of a certain color. The found piece 
of audio information, where the piece includes the audio hits, e.g., two pronounced words, 
matching the audio query, may be reproduced. In this case, tiie audio piece has longer in time 
than the audio hits. The audio hits may be recognized in the audio piece by, for example. 
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sfpedal audio markings like pre-determined sounds corresponding to ^'a beginning of fhe 
found hif ' and "an end of fhe found hif \ 

The use of fhe veib to comprise' and its conjugations does not exclude the 
presence of elements or steps other than those defined in a claim. The invention can be 
inoqplemented by means of hardware conqxrising several distinct elements, and by means of a 
suitably programmed computer. In tihe device claim enumerating several means, several of 
these means can be embodied by one and Hie same item of hardware. 
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CLAIMS: 



1 . A mefhod of ptesentiBg results of searching a document (200, 300), the search 

result conqirises a hit m the document, the method comprises the steps of: 

(130, 140) presenting at least one hit (230, 240) in a part of the document 
(210, 310) displayed on a screen, and 
5 - (180, 190) subsequently presenting at least one further hit (230, 240) in a 

further part of the document (220, 320) displayed on the same screen, at least one further hit 
(230, 240) being not comprised in said part of the document (210, 310) which has been 
displayed 

10 2. The method of claim 1 , wherein the further part of the document excludes 

search results which have been presented. 

3. The method of claim 1, wherein said document is one of an electronic, text 
document (200, 300), a data structure, video or audio information. 

15 

4. The method of claim 1, wherein the document is presented page-wise, and the 
part of the document conesfponds to apage entirely displayable on die screen. 

5. The method of claim 1, wherein the hits are visually marked on the screen. 

20 

6. The method of claim 1, further comprising a step of replacing an element of 
the document corresponding to a particular hit presented on the screen, or a step of replacing 
simultaneously all elemmts of tiie document corresponding to all hits presented on tiie 
screen. 

25 

7. The method of claim 1 , further comprising a step of aligning the presentation 
of the hits upward, downward, left or right to allow for a mayinmtn of the hits simultaneously 
shown on the screen. 
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8. The method of claim 1 , wherein at least two search queries are obtaiBed for 

searching the document, and the parts of the document to be displayed are correspondingly 
selected to present search results for one or more of the search queries. 

5 9. An iqpparatus (400) for presenting results of searching a document (200, 300), 

the search results comprise at least one hit in the document, the apparatus comprising a 
display means (440) coi:qpled to a processor (410) for enabling the sqpparatus: 

to present search results (230, 240) in a part of the document (210, 310) 
displayed on a screen, and 

10 - subsequently to present at least one further hit (230, 240) in a ftutihier part of 

the dociunent displayed on the same screen, wherein the further part of Ihe document (220, 
320) excludes the search results which have been presented, at least one ftcrther hit (230, 240) 
being not corcqprised in said part of the document (210, 310) which has been displayed. 

IS 10. A consumer electroiucs product being one of a TV set, a home cinenmsystesa, 

a portable video player, a remote control unit or a mobile phone, the product comprising an 
q^yparatus as claimed in claim 9. 

11. A computer program product enabling a programmable device when executing 

20 said computer program product to function as the apparatus as defibtied in claim 9. 
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ABSTRACT: 



The invention relates to a method of presenting results of searching a 
document (200, 300). Occurrences (230. 240) found in a part of the document (210, 310) are 
presented to iiie user on one screen. In a next step, a further part of the document (220, 320), 
comprising at least one fiirflier hit (230, 240) which has not been presented, may be displayed 
in flie same screen. Thus, all occurrences found in the document are presented to the user 
only once. 
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